Digital pen with enhanced educational feedback

ABSTRACT

Disclosed is a digital pen that tracks the user&#39;s writing and provides useful feedback based on the user&#39;s writing. In one embodiment, the pen may provide feedback when the user has written a misspelled word, invalid mathematical expression, or any noncompliant expression. The pen may also provide feedback relating to the user&#39;s handwriting. The feedback may be visual, auditory, or tactile, and may be realtime or delayed. Statistics relating to the user&#39;s performance may be tracked, uploaded to external devices, and shared with others. This allows the user and interested parties to track the user&#39;s progress over time. The disclosed pen will be useful in educational settings.

BACKGROUND

Digital pens are known in the art. These devices typically look, feel,and operate similar to a normal pen, but include electronics thatprovide additional functionality. Some digital pens have sensors capableof tracking movement of the pen tip to capture what the user is writing.These pens may be configured to record the user's pen strokes and storea digital version of the user's writing. Educational systems involvinghandwriting recognition are also known in the art. Information relatingto digital pens and educational systems can be found in the followingU.S. Patent and Patent Application Publication Numbers, the teachings ofwhich are hereby incorporated by reference: U.S. Pat. No. 6,831,632 toVardi; 2005/0125228 by Samuels; U.S. Pat. No. 7,120,385 to Marggraff;U.S. Pat. No. 7,175,095 to Pettersson et al.; U.S. Pat. No. 7,853,193 toMargraff; U.S. Pat. No. 8,038,538 to Stucke, Jr. et al.; 2012/0029907 byLoggins et al.; U.S. Pat. No. 8,427,344 to Margraff et al.; and U.S.Pat. No. 8,638,319 to Edgecomb et al.

U.S. Pat. No. 6,831,632 to Vardi discloses a hand-held handwritingrecognition device that may perform arithmetic computations based oninformation received from the handwriting recognition circuitry anddisplay the result. U.S. Pat. No. 8,427,344 to Marggraff et al.discloses a digital pen that may interpret the user's writing anddisplay a spelling correction. While these systems may be useful incertain contexts, they are of limited value in an educational setting.

SUMMARY

One aspect of the disclosure is a digital pen that tracks the user'swriting and provides context-appropriate feedback in real time or on adeferred basis. Another aspect of the disclosure is an educationalsystem that provides the ability to track the user's progress over timeand modify a scoring regime accordingly to further the user'seducational goals. The disclosed pen can enhance educational exercisesby providing instantaneous feedback to the user. Providing feedback inreal time may enhance the educational utility of the pen by drawing theuser's attention to the mistake as soon as it is made, rather thanallowing the mistake to go unnoticed and become reinforced by repetitionthroughout a writing or exercise. In some embodiments, the feedback mayalert the user to a mistake without revealing an answer, solution, orother correction. In this way, the pen may be especially useful ineducational settings. Additionally, the educational system can providegains to educators by providing new and more efficient modes of trackingtheir students' progress. For example, the system may provide educatorsor mentors with an interface for monitoring the performance of severalpupils in real time, so that the educator's attention can be directed tothe pupils most in need of assistance.

One aspect of the disclosure provides a writing device. The writingdevice may include a tip capable of writing on a surface and a sensorcapable of capturing a representation of the writing. The device mayalso include a processor coupled to the sensor that is capable ofgenerating compliance data based on the representation of the writingand one or more rules. Additionally, the writing device may include anoutput device coupled to the processor. The output device may be capableof generating feedback based on the compliance data. Furthermore, thewriting device may include a memory coupled to the processor. The memorydevice may be capable of storing at least a portion of the compliancedata.

Another aspect of the disclosure provides a method of providingeducational feedback. The method may comprise the step of creating awriting on a surface with a tip, and capturing a representation of thewriting with a sensor. The method may further include the step ofgenerating compliance data based on the representation of the writingand one or more rules.

The method may further include the step of generating feedback based onthe compliance data using an output device. Additionally, the method mayinclude the step of storing at least a portion of the compliance data ina memory.

Yet another aspect of the disclosure provides an educational system. Thesystem may include a communication device that receives compliance datafrom one or more handheld writing devices. The system may also include adisplay device that displays information related to the receivedcompliance data. The compliance data may be generated by a processorbased on one or more rules and a representation of a writing captured bya sensor coupled to each of the one or more writing devices.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a view of an exemplary writing device in accordance with thedisclosure.

FIG. 2 is a block diagram of the functional components of an exemplarywriting device.

FIG. 3A illustrates an exemplary writing device used in the context of amath problem.

FIG. 3B illustrates an exemplary writing device used in a spellingcontext.

FIG. 3C illustrates an exemplary writing device used in handwritingcontext.

FIG. 4 illustrates an educational system in accordance with thedisclosure.

FIG. 5 is a view of a user interface displayed in an exemplaryeducational system.

FIG. 6 illustrates another view of an exemplary user interface inaccordance with the disclosure.

FIG. 7 is a flow chart illustrating an exemplary process of a writingdevice in accordance with the disclosure.

FIG. 8 is a flow chart illustrating another exemplary process of awriting device in accordance with the disclosure.

FIG. 9 is a block diagram of an exemplary software application inaccordance with the disclosure.

DETAILED DESCRIPTION

Disclosed is an educational system that includes a digital pen capableof tracking the user's writing, determining whether the writing complieswith one or more rules, and providing feedback relating to the user'sperformance with respect to one or more goals. By way of example, Thepen may be used to track the user's performance in spelling, grammar,handwriting, mathematics, and other domains or activities. In someembodiments, the feedback may alert the user to a mistake withoutrevealing an answer, solution, or other correction. In this way, the penmay be especially useful in educational settings. The system may beupdated with new sets of rules pertaining to a variety of domains and/oractivities. The feedback may be real time or delayed. The system mayalso include an interface for tracking feedback from a multiple devices.The interface may allow a person, such as a teacher in a classroom, tomonitor feedback from two or more pens simultaneously and in real time.

Referring to the drawings more particularly by reference numbers, FIG. 1shows an exemplary pen 100 in accordance with the disclosure. The pen100 may include a handheld housing 105 composed of metal, plastic,glass, or other suitable material. At one end the housing may include atip 110 capable of writing on a surface. The tip 110 may be a fountaintip, ball-point tip, felt tip, or any other applicator designed to applya liquid ink to a surface. Alternatively, the tip 110 could be designedto apply other substances, such as graphite or wax, as with a pencil orcrayon. The housing 105 may also include a clip 115 for anchoring thepen 100 to a clipboard or a piece of clothing such as the breast pocketof a shirt or jacket. The housing 105 may further include one or moreopen or vented sections 120. The vented section 120 may facilitate thepassage of acoustic energy through the housing of the pen to and from amicrophone or speaker, or both, contained within the housing 105. Themicrophone and/or speaker (not shown) may be used for general recordingand playback of audio information. The microphone may also be used forreceiving voice commands for controlling the pen 100.

The pen 100 may also include a number of input and output devices forinterfacing with the user. For example, the pen 100 may include inputdevices such as buttons 125, pressure sensors 130, and microphones (notshown). The pen 100 may also include one or more sensors for receivinggestural inputs or input in the form of writing. For example, the pen100 may include one or more accelerometers or other motion sensors 135for tracking the orientation and movement of the pen 100 in space. Thepen 100 may include a first motion sensors or set of motion sensors forcapturing gestural input or general movement of the pen and a secondmotion sensor or set of motion sensors for capturing writing createdwith the pen tip. Alternatively, the pen may employ one more motionsensor or set of motion sensors to capture both types of input. Althoughthe one or more motion sensors 135 are illustrated as being near themidpoint of the housing 105, in other implementations, the motionsensors 135 may be disposed further towards either or both ends of thehousing 105. The pen 100 may also include one or more cameras or otheroptical sensors 140 for tracking the movement of the pen tip 110 withrespect to a surface. The optical sensor 140 may be a camera or otherimage sensor disposed near the tip of the pen that has a view of thewriting surface when the tip is engaged with the writing surface. Thewriting surface may bear naturally occurring surface features whosemovement can be tracked by the optical sensor. Alternatively, thewriting surface may bear an artificially created pattern that allows thesystem to track the position of the tip with respect to the writingsurface. An example of such a surface is described in U.S. Pat. No.7,175,095, incorporated by reference herein.

Alternatively, or additionally, the motion of the pen may be trackedusing one or more accelerometers or other motion sensors. An example ofsuch a system is described in U.S. Pat. No. 6,831,632, incorporated byreference herein. By way of example, the pen may include as many asthree or more independent accelerometers, each arranged to measuremotion along a different spatial axis. The pen may employ both opticaland motion sensors and intelligently fuse the output of both sets ofsensors to improve the reliability of the handwriting detection andextend the overall functionality of the pen. The pen may treat motiondetected by an accelerometer differently when it is accompanied bymotion of the writing surface detected by the optical sensor. Forexample, the pen may disregard motion detected by an accelerometer whennot accompanied by motion of the writing surface detected by an opticalsensor. Alternatively, the pen may treat such detected motion as aninput for changing the operational mode of the pen rather than a writingto be stored or interpreted by the processor. For example, shaking thepen, i.e., moving the pen rapidly back and forth or side-to-side, awayfrom a writing surface may toggle the device power, an audio recordingfunction, a handwriting recognition and recording function, or otherfunctions. However, moving the pen back and forth while the tip isengaged with a writing surface may be treated as a gesture of strikingout text, and may cause a recent unit of writing to be disregarded bythe pen's performance tracking algorithm, or deleted from the pen'shandwriting detection and recording function. Detecting the presence ofa writing surface under the pen tip could alternatively or additionallybe accomplished using a sensor that measures pressure on, or deflectionof, the pen tip.

The pen 100 may additionally include one or more pressure sensors 130disposed in, on, or around housing 105. The one or more pressure sensors130 may be used to detect whether and how the user is gripping the pen.For example, the pen may be configured to transition to an active statewhen sensors 130 indicate that the pen is being gripped or held.Additionally, the pen 100 may disregard or treat differently movement ofthe pen detected when the sensors 130 do not indicate that a user isgripping the pen in a usual manner. Additionally, the pen 100 could beconfigured to provide feedback to the user when values from the sensors130 indicate that the pen 100 is being held inconsistently with adesired force/pressure profile. For example, the pen 100 may beep,vibrate, or illuminate an LED if the user is gripping the pen 100 tofirmly or too softly, or too far from or too close to the tip 110. Amulticolored LED may be used to display a range of colors to indicatehow severely the user's current grip deviates from the desired forceprofile.

The pen 100 may additionally include a number of output devices forproviding feedback and other information to the user. These may includelight sources, such as one or more LEDs 145, audio sources, such asspeakers or buzzers (not shown), and tactile feedback sources, such aselectromechanical oscillators that produce vibration (not shown). Theone or more LEDs 145 may be multicolored to communicate feedback ofdifferent types and/or degrees. Although FIG. 1 illustrates that the oneor more LEDs 145 are disposed at one end of the pen 100, it is to beunderstood that the one or more LEDs 145 may be disposed at either endof the pen 100, and/or anywhere in between. For example, it may beadvantageous to position one or more LEDs 145 near the tip 110 or nearwhere the pen is typically gripped to facilitate being seen by the user.The pen 100 may also include a display screen 150 for displaying variousinformation to the user, such as characters, words, or graphics, toprovide feedback or indicate a current mode of the pen. For example, ifthe pen were set to monitor and provide feedback regarding the user'sspelling, the display screen 150 may display the word “spelling” or theletters “ABC”. Additionally, display screen 150 may display various menuoptions. The user may navigate through these menu options using buttons125. Exemplary display screens 150 could include LED, LCD, or any othervisual display devices capable of displaying characters, symbols, words,or graphics. The various output devices allow the pen 100 to providefeedback to the user as well as communicate various information to theuser regarding the state of the pen 100 and other operationalparameters.

FIG. 2 is a block diagram illustrating various functional components ofan exemplary hardware configuration 200 of a pen in accordance with thedisclosure. The various components may communicate with each other viaone or more buses 280. The pen may include one or more batteries (notshown) or other power supply for powering the various electroniccomponents of the pen. The battery may be replaceable or rechargeable,or both. The pen may be configured to be wirelessly rechargeable. Thepen may also be powered by a wired or wireless connection to an externalpower supply.

As discussed above with respect to FIG. 1, the pen may include a numberof user input devices 210 such as buttons, switches, pressure sensors,etc., that allow the user to activate, deactivate, or otherwise changethe operational mode of the pen. The pen may also include sensors 220for capturing a representation of the user's writing and other movementsof the pen. The sensors 220 may include one or more optical sensors,accelerometers, gyroscopes, magnetometers, or other devices capable ofdetecting or measuring movement or motion.

The various sensors 220 and user input devices 210 may be coupled to aprocessor 230. The processor may execute application software asdiscussed in greater detail below. The application software may bestored on a non-transitory computer-readable medium, such as a randomaccess memory (RAM) 240, a mass storage device 250, a ROM, a FLASHmemory, an EEPROM, or any other suitable memory or storage device on thepen or an external device in communication with the pen. The processor230 may be physically attached to or incorporated within the housing ofthe pen, or may be disposed in an external device in communication withthe pen. In one exemplary mode of operation, the processor 230 mayreceive a representation of the user's writing in the form of signalsoutput from sensors 220. The processor 230 may perform handwritingrecognition on the output of the sensors 220. Once the handwritingrecognition has been performed, the processor 230 may parse therecognized handwriting using language processing algorithms to assembleindividual alphanumeric characters, symbols, punctuation marks, andmathematical symbols into words, phrases, sentences, paragraphs, andother grammatical units. For example, the processor 230 may recognize asequence of letters as an English word, such as “cat”. Similarly, theprocessor 230 may recognize a sequence of numbers interspersed witharithmetic operators as a mathematical expression. For example, thesystem may identify “2” followed by “+” and “2” and “=” as amathematical expression.

Depending on the operational mode of the pen, the processor 230 maygenerate compliance data by applying one or more rules to the recognizedwriting. The rules may pertain to spelling or grammar in a particularlanguage, mathematics, handwriting, or other domains. For example, thecompliance data may specify whether a word has been misspelled, agrammar rule violated, a letter formed in a manner inconsistent withhandwriting conventions, or whether an identified mathematicalexpression is invalid. The processor 230 may monitor the writing forcompliance with rules in multiple domains at once, or only a singledomain. The specific domain or domains for which the processor 230monitors compliance may be specified by user input provided via the penor an external device in communication with the pen. Alternatively, thedomain or domains may be specified automatically as part of aneducational program or algorithm.

The processor 230 may store all or a portion of the compliance data in amemory, such as a random access memory (RAM) 240. The contents of theRAM 240 may be transferred to a mass storage device 250. The processor230 may also calculate statistics related to the stored compliance data.These statistics may also be stored in the RAM 240, storage device 250,or communicated to an external device in communication with the pen viaa communication port 270.

The processor 230 may additionally be configured to store therepresentation of the writing and any recognized text in mass storagedevice 250. The mass storage device may be coupled to the housing of thepen, or may be disposed in an external device that can communicate withthe pen. The mass storage device 250 could additionally or alternativelytake the form of a removable thumb drive that mates with a communicationport 270 of the pen. The contents of the mass storage device 250 may betransferred or synced automatically with an external device. It is to beunderstood that the functions accomplished by the RAM 240 and the massstorage device 250 depicted in FIG. 2 could alternatively be performedby a single memory device.

The pen may include various output devices 260 coupled to the processor230 for providing feedback to the user. In one embodiment, when theprocessor 230 determines that the user has written a misspelled a word,the processor may illuminate an LED to alert the user to themisspelling. In another embodiment, the processor 230 may activate anelectromechanical oscillator to cause the pen to vibrate. In yet anotherembodiment, the processor 230 may cause a speaker on the pen to play anaudible tone or sound.

The pen may also include a communication port 260. The communicationport may support wired communication protocols, wireless communicationprotocols, or both. Exemplary wired communication protocols couldinclude USB or HDMI. Exemplary wireless communication protocols couldemploy Wi-Fi, such as any of the IEEE 802.11 standards, BLUETOOTH, orcellular protocols.

FIG. 3A illustrates an example of the pen 100 being used in anarithmetic mode. When the user writes “2+2=5”, the processor recognizesthe writing as a mathematical expression. The processor subsequentlyapplies arithmetic rules to the expression and determines the expressionto be invalid. Accordingly, the processor may activate an output devicesuch as a red LED to alert the user to the invalid expression.Additionally, the processor may activate a green LED had the userwritten a valid expression, such as “2+2=4”. FIG. 3B illustrates anexample of the pen 100 being used in a spelling mode. Here, theprocessor identifies “rid” as a misspelling. In some cases, the pen 100may simply identify misspellings by searching a dictionary of therelevant language for an identified word and determining whether theidentified word is present in the dictionary. However, in some cases, asillustrated in FIG. 3B, a word such as “rid” may be found in the Englishdictionary, even though it is a misspelling in context. Thus, in someembodiments, the pen 100 may also apply more sophisticated rules tosurrounding words, phrases, or sentences to determine whether any of therules is violated. For example, the pen 100 may function in a mode inwhich the user is asked to write sentences about a predefined scene orsituation, such as a picture or verbal description of a scene includinga red ball. The processor may recognize the text “The ball is” asintroducing an attribute of the ball. Thus, if “rid” is not found in alist of attributes associated with the ball (e.g., red, round, etc.),the pen 100 may illuminate an LED to indicate this noncompliance to theuser.

FIG. 3C illustrates an example of the pen 100 being used in ahandwriting mode. The pen 100 may maintain a set of rules relating toguidelines for forming the letters of the alphabet and otheralphanumeric symbols. If the pen 100 identifies either an unrecognizedcharacter or a recognizable character that was formed improperly, it mayprovide feedback to alert the user to the noncompliance.

FIG. 4 illustrates an educational system 400. The system 400 may includeone or more pens 410 such as the pen 100 shown in FIG. 1 and describeabove. The system may be set up in a classroom and may also include abase station 420 or wireless access point that forms part of a wirelesslocal area network (WLAN) 430 that allows communication between the pens410 and an external computing device 440, such as a laptop, desktop,tablet, or smartphone. One or more of the pens 410, the access point420, or the external computing device 440 may be coupled to the Internet450 or other communication network to allow communication with a remoteserver 460. The remote server may be operated by a school, educationalservice provider, or other organization. In one example, as one or morestudents in the class room use their pens 410, the pens 410 maywirelessly communicate the associated compliance data and otherinformation to the external computing device 440. Although externalcomputing device 440 is shown inside a classroom in FIG. 4, it is to beunderstood that the external computing device 440 could be locatedoutside the classroom as well. In general, external computing device 440can be located anywhere a connection to the Internet 450 is available.

The server may include a web application or interface that can bedisplayed by a web browser running on an external computing device 440.The web interface may enable any or all of the functionalities describedthroughout disclosure with respect to the external computer device 440.Additionally, or alternatively, the external computing device 440 mayexecute its own software application that provides any or all of thesefunctionalities.

A user of the external computing device 440, such as a teacher in aclassroom or an administrator at the educational service provider, mayuse an interface of the external computing device 440 to define a groupof pens. The group of pens may be a subset of all the pens with whichthe external computing device 440 or the remote server 460 cancommunicate. The group of pens may include pens in disparategeographical locations, such as different classrooms or differentschools. Additionally, for example, the group may include individualpens associated with students in arbitrary locations that areparticipating in an online course. In one embodiment, the user may useexternal computing device to configure the group of pens in a specificmode, or for a specific assignment or exercise. For example, the usermay configure the group of pens for a spelling test. In this example,the group of pens may be set to monitor and provide feedback regardingspelling for a specific period of time, or until a specified number ofwords have been written, or a specified number of correctly orincorrectly spelled words have been written. Similar tests could bedefined for math, handwriting, and other domains. At the conclusion ofthe test, the pens may transmit scores or compliance data to the remoteserver 460 or the external computing device 440. In one example, thecompliance data received from the pen may be transmitted to the remoteserver 460, which may subsequently transmit the received information tothe external computer device 440. Alternatively, the remote server 460may transmit to the external computing device 440 only scores or otherderivative information based on the information the remote server 460receives from the pens.

Additionally, the pens may communicate the entirety of the capturedwriting to the external device 440 or the server 460. This may eliminatethe need for students to hand in a hard copy of a given assignment orexercise. Handwriting verification may be employed to verify that thehandwriting captured by a particular pen matches a handwriting profileof the user ID associated with the pen during the assignment. Theinformation from the pens may be communicated on a real-time or deferredbasis. For example, the information may be transmitted continuously,upon completion of a predefined exercise or assignment, upon expirationof a predefined time interval, at a specific time, upon input to the penby the student, or in response to a request initiated by the externalcomputing device 440. For example, the external computing device mayrequest information from registered pens periodically, at a specifictime or upon a specific event, such as opening or closing of anapplication on the external device, or upon input from the teacher orother user via the interface.

In one example, one or more pens 410 may be networked together to permita group of people to engage in a timed exercise or competition. The pens410 may communicate with each other, an external computing device 440,or a server 460 to determine which user achieved a highest score for thecompetition. At the conclusion of the competition, the winning user mayreceive positive feedback from his or her pen, and other users mayreceive negative feedback from their pens. Additionally, oralternatively, the pens 410 may provide real time feedback during thecompetition to indicate to each user their current rank in thecompetition. For example, the pens 410 may illuminate a green LED if theuser is currently leading in the competition, and a red LED if the useris currently trailing in the competition. The pens 410 my blink the LEDswith increasing or decreasing frequency as the user's lead or deficitincreases or decreases during the competition. Users may also use thesystem 400 to participate in asynchronous competitions, in which eachuser's input is received at different times and individual scores orcompetition results are displayed on a webpage or other interfaceavailable via an external device 440.

FIG. 5 illustrates an exemplary screen 500 of a graphical user interfacethat may be displayed on an external computing device. In one example,the interface may be used by a teacher in the classroom to monitor theperformance of one or more students using a pen in accordance with thedisclosure. In one embodiment, the interface may include an area 510that displays an identifier associated with each pen registered with theinterface. The user may select a particular ID to display furtherinformation associated with the selected ID as discussed further withrespect to FIG. 6. The interface may also include an area 520 thatdisplays a score or metric associated with the compliance data receivedfrom each pen. The external computing device may receive compliance datafrom the pens and calculate the score or metric itself. Alternatively,the score or metric may be computed at the pen or other device, such asa remote server, and communicated to the external computing device fordisplay on the interface. The interface may also display otherinformation, such as an indication of the user's current activity levelor category, the operational mode of the pen, device status, or otherinformation. The interface may be configured to highlight, color-code,or otherwise display a call out 530 to individual scores depending onwhether they exceed or fall below a specified threshold, or otherwisefall within various ranges. Additionally, the interface may beconfigured to automatically sort the list of scores from highest tolowest, or lowest to highest, so that a teacher or instructor may bemade aware of students that are performing especially well or poorly.The sort order may be updated in real time if one or more of the pensare active. Alternatively, the screen may display or highlight the nameof a user who has made several mistakes in succession or within aparticular time window. The time window may be adjustable via theinterface. In another example, a parent may use the interface on theirsmartphone to remotely monitor the progress of a child as he or shecompletes a homework assignment or practices a particular activity. Inanother example, in addition to tracking compliance, the interface maycompute statistics pertaining to how long each student takes to completeeach individual problem. For example, the list may highlight studentstaking especially an especially long or short period of time to completecorrect mathematical expressions, such as “2+2=4”.

FIG. 6 illustrates another exemplary screen 600 that may be displayed bya graphical user interface in accordance with the disclosure. The screenmay be displayed by selecting a particular ID listed on the screen 500displayed in FIG. 5. The screen 600 may include areas for displaying avariety of information, including the selected ID 610, the current ormost recent activity for which the pen was used 620, a score or othermetric 630 related to compliance data generated by the pen, recentwritings 640 or other input recorded by the pen, and/or informationrelating to the compliance of individual inputs recorded by the pen. Thescreen 600 may display an indication 650 of whether recent inputcomplies with one or more compliance rules. Although symbols 650 areillustrated as the letter “x” and a check mark in FIG. 6, the screen 600could employ alternate ways of indicating compliance of individualinputs, such as by varying fonts color, size, or type face, backgroundcolor, highlighting, or using animation. The screen 600 may also includevarious selectable features for refreshing the data received from thepen, requesting additional data from the pen, sending a message to thepen, configuring the pen, and/or displaying one or more storedrepresentations of writing recorded by the pen. In an anotherembodiment, the user may swipe or select an icon on screen 600 todisplay a full-page rendering of the captured writing received from aparticular pen or ID. In this embodiment the, the interface mayunderline, highlight, or otherwise visually call out portions of textthat violate compliance rules, as well as display numerical scores orstatistics related the captured writing. The pen may have a specifiedinput, such as a button or verbal command or gesture that signals thebeginning of a new page. Alternatively, the user's writing may berecorded on a continuous document, such as a scroll. The interfacedisplayed on the external device may allow the user to page or scrollthrough the recorded writing. The screen 600 may also be configured todisplay a list of domains or activities and a score or metric indicatingthe user's performance in each domain or activity. This may be usefulfor quickly identifying the user's strengths and weaknesses.

FIG. 7 is a flowchart illustrating an exemplary process 700 carried outby the pen. The process 700 may be executed by a processor on the pen oron an external device. Alternatively, execution of the process 700 maybe divided across a processor of the pen and a processor of an externaldevice. At step 710, as the pen is used to write on a surface, one ormore sensors capture a representation of the writing by detecting themovement of the pen. At step 720, the processor analyzes therepresentation of the writing using handwriting recognition algorithms.The output of the handwriting recognition step is a sequence ofalphanumeric characters, symbols, punctuation marks, etc. The processoranalyzes the recognized characters and symbols using language andgrammar processing algorithms to identify individual letters and symbolsas well as assemble words, phrases, sentences, paragraphs, equations,and other units of grammar.

As the processor decomposes the representation of the writing intogrammatical units, at step 730 the processor applies one or more rulesto each grammatical unit. For example, if the pen is set in a spellingmode, the processor will verify whether a word written with the penexists in a dictionary or otherwise fits its context, as discussed abovewith respect to FIG. 3B. At step 740, the processor generates compliancedata. For example, in the spelling example, if a particular written worddoes not appear in the dictionary, the processor will generate negativecompliance data, indicating that no match was found. At this point, instep 750, the processor may generate feedback to alert the user of theunrecognized word by illuminating an LED on the pen, playing a sound, orcausing the pen to vibrate. At 760, the processor may store all or asubset of the compliance data generated in step 730. For example, thepen may store only negative compliance data, i.e., data regardingnon-compliant writings. Although the above steps are described as beingperformed in a specific sequence, it is contemplated that some stepscould be performed in another order or performed in parallel.

The user may activate or deactivate the above-described process byproviding input to the pen or an external device in communication withthe pen. The input could include manipulating a button or switch,speaking an audible command that can be detected by a microphone, orperforming a specific gesture, such as shaking the pen or tapping thepen on a surface. Because the pen may have more than one user, beforethe above-described process begins, or as part of the above-describedprocess, the user may be required to provide a user ID, so that thecaptured representation of the writing, compliance data, performancestatistics, and other information generated or recorded while the pen isin use may be associated with a particular user. The user may providetheir ID via an external device in communication with the pen, or bywriting their ID, a PIN, or both, on a writing surface, or by usingbuttons or other input devices on the pen to select an appropriate IDfrom a list of IDs stored in the pen, and in some cases subsequentlyprovide a PIN to verify their identity. The user could alternativelyspeak their ID and/or PIN into a microphone of the pen, or voicerecognition software could be used to analyze the user's voice and matcha specific ID associated with the user's voice. Alternatively, the pencould include a thumbprint or other biometric scanner that could be usedto identify the user. In another embodiment, the pen may storehandwriting profiles for multiple users and employ handwritingrecognition software to identify the current user after the user beginswriting.

FIG. 8 is a flow chart illustrating another exemplary process 800carried out with a writing device in accordance with the disclosure. Theprocess 800 may be executed by a processor on the pen or on an externaldevice. Alternatively, execution of the process 800 may be dividedacross a processor of the pen and a processor of an external device. Theprocess functions to modify the pen's behavior as the user's performanceprogresses or regresses. At step 810, as the pen is used, compliancedata is generated based on one or more rules as described above. At step820, the processor monitors one or more statistics relating to thecompliance data. At step 830, if the processor determines that thestatistic does not exceed a specified threshold, then the processreturns to step 810. If, however, at step 830 the processor determinesthat the statistic does exceed the specified threshold, the processormay at step 840 update or change the one or more rules used ingenerating the compliance data before the process returns to step 810.In this way, the pen or external device may modify its behavior as theuser's performance changes over time. By way of example, if the userwere a young child learning handwriting, the pen may not providereal-time feedback for misspelled words, but rather only improperlyformed characters, e.g., a backwards “e” or “j”, etc. However, as theuser's handwriting performance improves, and fewer handwriting mistakesare made, the pen may begin to generate feedback for misspelled words.Additionally, as the user's spelling performance advances, the pen maybegin to generate feedback based on more advanced rules regardinggrammar, punctuation, style, etc. Alternatively, rather than changing orupdating compliance rules, the processor may produce compliance datausing all available rules and simply update one or more feedback rulesthat govern when, how often, and what type of feedback is generated.

FIG. 9 is a block diagram of a software application 900 in accordancewith the disclosure. The application 900 may be stored on anon-transitory computer-readable medium. The medium may be disposed onthe pen or an external device in communication with the pen, or both.The medium may be a random access memory (RAM), read-only memory (ROM),mass storage device, FLASH memory, EEPROM, or any other suitable memorydevice. The application 900 may include a number of functional modulesthat implement the various features and functionalities of the pen. Theapplication 900 and its various modules may execute on a processor onthe pen or on an external device in communication with the pen, or both.The application 900 may include a capture module 910 that takes in datafrom the one or more sensors on the pen used to capture a representationof the user's writing. Handwriting recognition module 920 may receivethe representation of the writing from capture module 910 and analyzethe representation to identify characters and symbols in therepresentation of the writing received from capture module 910.Compliance module 930 may receive the characters and symbols fromhandwriting recognition module 920 and apply grammar or language rulesto assemble the individual characters and symbols into grammatical unitssuch as words, phrases, clauses, sentences, paragraphs, mathematicalterms, expressions, etc. Compliance module 930 may then apply one ormore rules to the various grammatical units to determine whether aparticular unit complies with, or violates, any of the one or morerules.

The compliance module 930 may output compliance data to a feedbackmodule 940. Feedback module 940 may use the compliance data to determinewhether and/or how to provide feedback to the user. For example, thefeedback module 940 may examine stored user settings or configurationinformation to determine whether to provide feedback based on thereceived compliance data. Alternatively or additionally, the feedbackmodule 940 may examine the users performance to determine whether toprovide feedback. In other embodiments, whether and how the deviceprovides feedback may depend on the particular educational programcurrently executing on the pen or external device. Alternatively, thefeedback module 940 may always provide feedback.

The feedback module 940 may also use configuration information, usersettings, or an algorithm to determine how or what type of feedback toprovide. For example, the user may set the device to provide visualfeedback, such as illuminating an LED. The user may alternatively oradditionally set the device to provide audible feedback, such as playinga tone, and/or tactile feedback, such as vibration. Additionally, theuser may configure the device to provide no feedback, or to providefeedback only when the user indicates a desire to receive feedback. Forexample, the user may push a button on the pen to indicate a desire forfeedback. Alternatively, the device may detect that the user hasperformed a specified gesture, such as shaking or tapping the pen, toindicate that he or she desires feedback.

The feedback module 940 may additionally be configured to providedifferent types of feedback to enhance the educational value of the pen.For example, the pen may defer feedback depending on the type of problemand provide a verification mode that allows the user to check theirwork. In one example, the user writes out a multi-step math problem. Atthe end of the problem, the pen may illuminate a red LED to indicatethat the solution written is incorrect. The pen may then automaticallyenter a verification mode, during which the user begins tracing over orre-working the problem from the beginning. The pen may illuminate agreen LED to indicate compliance for each individual step of theproblem, and then illuminate a yellow or red LED to indicatenon-compliance. In another example, after misspelling the word “yellow”as “yellow”, the pen may illuminate a red LED. After entering theverification mode, the pen may illuminate a green LED after the userwrites each of the letters “y”, “e”, “l” and then a yellow or red LEDwhen the user writes the letter “o” rather than a second “l”. In thisway the user is made aware that the “o” is incorrect. In anotherembodiment, the pen may register the position of the compliance datarelative to the writing surface where the compliance data was generated.The pen may then produce positive or negative feedback as it the tip ismoved into proximity of the various positions on the writing surfacewhere compliance data was generated. The position data may be generatedusing an optical sensor and specially marked paper. In an alternateembodiment, the compliance data may be registered to positions in spaceas measured by one or more motion sensors or inertial navigation units.

The application 900 may also include a communication module 950 forcommunicating with external devices. For example, the communicationmodule 950 may employ one or more standard communication protocols suchas Wi-Fi, USB, or Bluetooth to allow the pen to communicate with variousexternal devices such as wireless access points, laptop computers,desktop computers, printers, smartphones, headsets or earphones, etc.The communication module 950 may also enable the pen to communicate withexternal servers via the Internet. For example, the communication module950 may enable access the Internet to allow the user to upload theirperformance information to an educational service provider.Additionally, the pen may access the Internet to automatically uploadthe stored representation of the user's writing to be archived on aserver. The communication module 950 may also enable access to theInternet or other external devices to receive software updates, firmwareupdates, patches, upgrades, or new programs. Additionally, thecommunication module 950 may enable the pen to network with other pens.For example, the communication module 950 may allow user data, such ascaptured representations of the user's writings, compliance data,performance statistics, as well as profile information such as user IDand preferences, etc., to be transferred directly to another pen.

The application 900 may additionally include a storage module 960 formanaging the storage of user data, such as captured representations ofthe user's writings, compliance data, performance statistics, profileinformation, and other information. The storage module 960 may work inconjunction with the communication module 950 to manually orautomatically back up the stored information to an external device.

The application 900 may also include a rules module 970. The rulesmodule may contain one or more rules used by the compliance module 930or the feedback module 940 to generate compliance data and feedback. Therules maintained in the rules module 970 may be updated automatically ormanually. For example, the rules may be updated when there is a softwareupdate or upgrade, when the user adds new software or changes theoperational mode the pen, or when the pen syncs with an external devicethat has permission to update the rules maintained by the rules module970. As an additional example, the rules may be updated by an algorithmthat selectively updates the rules depending on the user's performance.

The application 900 may additionally include an input module 980. Theinput module 980 may receive, process, or respond to informationreceived from the one or more input devices associated with the pen. Forexample, the input module 980 may receive audio information from themicrophone and process the audio information using voice recognitionsoftware to determine whether the user has issued a command to which theinput module 980 should respond. If, for example, the user spoke thewords “stop feedback”, the input module 980 may then communicate withthe feedback module to stop generating feedback. Alternatively, theinput module 980 may stop feedback by communicating with an outputmodule 990 to disable one or more feedback or output devices. The inputmodule 980 may also receive, process, or respond to information receivedfrom buttons and other sensors.

The application software may also include an output module 990. Theoutput module may 990 control the various output devices of the pen. Forexample, in response to a communication from the feedback module, theoutput module 990 may generate an audio signal via a speaker in order togenerate audio feedback. Additionally or alternatively, the outputmodule 990 may cause an LED to illuminate, an electromechanicaloscillator to vibrate, or a display on the pen to display a particularmessage to the user.

Although the application 900 has been described as including specificmodules for performing specific tasks, this description is intended tobe illustrative only. It is to be understood that any and all of theabove-described functionalities may be carried out by more, fewer, ordifferent software modules.

The pen may be configured to teach a wide variety of subjects useful atvarious stages of learning and intellectual development. For example, inone embodiment, the pen may be configured to identify and classifyobjects, pictures, or diagrams drawn with the pen. For example, if theuser were to draw a picture of a flag followed by an “=” and then thenwrite the word flag, the pen may illuminate a green light or otherwiseindicate positive feedback. Similarly, the pen could be configured tooperate in a language translation mode. For example, in this mode, theuser could specify an English/Spanish mode. Subsequently, if the userwere to write “cat” followed by an “=” and the word “gato”, the penwould provide positive feedback.

Yet another example includes science. In this example, the pen could beconfigured to provide positive feedback after the user writes correctscientific relations or identities. For instance, if the user were towrite a chemical formula, such as “H₂0” followed by an “=” and the nameof the chemical, i.e., “water”, the pen may illuminate a green LED. Yetanother example includes history, wherein the pen may be configured toprovide positive feedback after writing “#1president=George Washington”or “George Washington was the first president of the United States”. Ingeneral, the pen may employ language processing algorithms to providefeedback indicating the veracity of any recognizable identity, equation,relation, statement, or formulation that can be verified againstavailable databases. Portions or all of the language processing anddatabase queries may be performed on one or more external devices incommunication with the pen.

Additionally, a user may upload all of his or her captured writings toan external device. The writings may be indexed by date, time, location,and/or pen to facilitate search and retrieval of specific writings frommultiple pens. The stored writing may also be text searchable.

While certain exemplary embodiments have been described and shown in theaccompanying drawings, it is to be understood that such embodiments aremerely illustrative of and not restrictive on the broad invention, andthat this invention not be limited to the specific constructions andarrangements shown and described, since various other modifications mayoccur to those ordinarily skilled in the art.

I claim:
 1. A writing device, comprising: a tip capable of writing on asurface; a sensor capable of capturing a representation of the writing;a processor coupled to the sensor, the processor capable of generatingcompliance data based on the representation of the writing and one ormore rules; an output device coupled to the processor, the output devicecapable of generating feedback based on the compliance data; and, amemory coupled to the processor, the memory device capable of storing atleast a portion of the compliance data.
 2. The device of claim 1,wherein generating compliance data includes identifying a misspelledword.
 3. The device of claim 1, wherein generating compliance dataincludes identifying an invalid mathematical expression.
 4. The deviceof claim 1, wherein generating compliance data includes identifying animproperly formed character.
 5. The device of claim 1, wherein thesensor includes at least one of one of an optical sensor and anaccelerometer.
 6. The device of claim 1, wherein said feedback is atleast one of audible, visual, and tactile.
 7. The device of claim 1,further comprising a communication port coupled to said processor, thecommunication port capable of receiving one or more rules used by theprocessor in generating compliance data.
 8. The device of claim 7,wherein said communication port is further capable of transmitting saidcompliance data to an external device.
 9. The device of claim 8, whereinsaid communication port is further capable of receiving compliance datafrom an external device.
 10. The device of claim 1, wherein theprocessor is configured to: generate compliance data based on a firstset of rules when a statistic related to the compliance data is within athreshold; and generate compliance data based on a second set of ruleswhen said statistic exceeds said threshold.
 11. The device of claim 1,wherein the stored compliance data are associated with a useridentification (ID).
 12. The device of claim 12, further comprising auser input device, and wherein the processor receives a user ID via theuser input device.
 13. A method of providing educational feedback,comprising: creating a writing on a surface with a tip; capturing arepresentation of the writing with a sensor; generating compliance databased on the representation of the writing and one or more rules;generating feedback based on the compliance data using an output device;and, storing at least a portion of the compliance data in a memory. 14.The method of claim 11, wherein generating compliance data includesidentifying a misspelled word.
 15. The method of claim 11, whereingenerating compliance data includes identifying an invalid mathematicalexpression.
 16. The method of claim 11, wherein generating compliancedata includes identifying an improperly formed character.
 17. The methodof claim 11, further comprising transmitting at least a portion of saidcompliance data to an external device.
 18. The method of claim 11,further comprising: generating compliance data based on a first set ofrules when a statistic related to the compliance data exceeds athreshold; and generating compliance data based on a second set of ruleswhen said statistic does not exceed a threshold.
 19. The method of claim11, wherein said compliance data is associated with a useridentification.
 20. An educational system comprising: a communicationdevice that receives compliance data from one or more handheld writingdevices; a display device that displays information related to saidcompliance data, wherein said compliance data is generated by aprocessor based on one or more rules and a representation of a writingcaptured by a sensor coupled to each of said one or more writingdevices.